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TWO-STEP COMMERCIAL RECOMMENDATION 

The invention relates generally to commercials in audio and/or video signals such 
as television or radio signals and, more particularly, to a method and apparatus for 
personalizing the commercials in such signals for a user. 
5 In order to improve the user's experience, and to make commercials more effective, 

one can replace the commercials in a live broadcast stream with personalized commercials. 
U.S. patent 6,177,960 to Van Luyt for a *TV signal receiver", issued Jan. 13, 2001, and 
incorporated herein by reference, discloses one possible scheme for replacing the 
commercials in a live broadcast stream with personalized commercials. For example, the 

10 personalized commercial can be selected based on the demographic factors of the target 
audience of a particular TV or radio program. Such factors may include, e.g., gender, age, 
income and geographic location, and may be predicted based on the content of the program 
using known research and survey techniques. Advertisers can then choose to advertise 
their products on particular programs whose demographic factors correlate with those of 

15 the product of service being offered. This correlating, however, is much too rough to 

provide tmly personalized results since the interests of each individual can vary widely in 
ways that cannot be predicted by the demographic factors. 

Accordingly, it would be desirable to provide a method and apparatus for 
personalizing the commercials that are displayed to a user in commercial breaks of a video 

20 and/or audio program. 

In a particular aspect of the invention, a method is provided for selecting 
personalized commercials. The method includes providing, for each of a plurality of 
programs, a score indicating a degree of preference of at least one user in relation thereto; 
providing, for each of a plurality of commercials, respective correlation factors indicating 

25 respective degrees of effectiveness in relation to each of the plurality of programs; and 
providing, for each of the plurality of commercials, a metric indicating a degree of 
effectiveness in relation to the at least one user based on the scores and the respective 
correlation factors. The at least one user may be, e.g., an individual person, or a group of 
people in a household. 

30 A related apparatus and program storage device are also provided. 

In the drawings: 
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Fig. 1 illustrates an embodiment of an apparatus for recommending commercials; 
Fig. 2 illustrates linking a user to programs and commercials; and 
Fig. 3 illustrates an embodiment of a method for recommending commercials. 
In all the Figures, corresponding parts are referenced by the same reference 
5 numerals. 

The present invention involves making commercial recommendations much more 
personalized. To this end, a two-step linking between a user and a set of commercials is 
provided. The first step is to determine the relation between the user and a set of programs, 
such as television or radio programs, or other video, audio or audio/video programs. This 

10 can be achieved, e.g., using a program recommender. This relation indicates how much the 
user likes each of the programs. The second step is to determine the relation between the 
set of programs and the set of commercials. This can be achieved, e.g., based on the 
advertiser's knowledge. Then, the link between the user (e.g., viewer/listener) and each of 
the commercials can be determined based on the relations between the user and the 

15 programs, and the relations between the programs and the commercials, to identify one or 
more personalized commercials for the user. 

Fig. 1 illustrates an embodiment of an apparatus for recommending commercials. 
In one possible approach, the invention is implemented using components within a 
television set-top box receiver that receives a television signal and outputs a signal for 

20 display on a television. However, the invention is generally applicable to any type of 
device that receives video programs, audio programs, or audio/video programs. For 
example, the invention may be implemented in a computer that receives audio/video 
programs fi-om a network such as the Intemet, e.g., by downloading, streaming or 
broadcast. The video programs typically include an audio track although this is not 

25 required. Audio-only programs may include the audio track of a radio program, for 

example. Generally, the programs may be provided by any source, including the Intemet, 
cable, and terrestrial or satellite broadcasts. The programs may include pay-per-view 
programs. The programs may be played as they are received, or stored for subsequent 
playing. 

30 The present example refers to a video program for illustration only. In one 

approach, the receiver 100 demultiplexes and decodes the received video programs at a 
demultiplexer/decoder 110. The video programs may be provided in a digital or analog 
multiplex that is transmitted by cable, satellite, or terrestrial broadcast, for example. 
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Generally, one of the video programs is decoded based on a channel selection made by the 
user/viewer via a user interface 130. The decoded video program may be communicated to 
a display device 190 via a CPU 140, which includes a working memory 150, or stored 
locally for subsequent display, e.g., at a video storage device 1 15. In one possible design, 
5 the working memory 1 50 is a program storage device that stores software that is executed 
by the CPU 140 to achieve the functionality described herein. However, resources for 
storing and processing instructions such as software to achieve the desired ftmctionality 
may be provided using any known techniques. 

A commercial storage device 120 stores a number of commercials, which may be 

10 received at the receiver 100 using any known technique. For example, the commercials 
may be received and stored over time with the broadcast of the video programs. The 
commercials may be received via the same or a separate communication path from which 
the programs are received. Particular ones of the commercials are selected for insertion 
into commercial breaks in the video programs using the techniques disclosed herein. The 

15 personalized commercial insertion can use the local commercial storage 120 to play out a 
commercial. Multiple commercials in a broadcast signal may be another way to offer a 
choice of commercials to the viewers. A dedicated channel may carry commercials from 
which selections can be made. 

Various techniques for inserting personalized commercials into video programs are 

20 described in, e.g., WO 98/36563 to Van Luyt, published August 20, 1998, entitled "TV 

signal receiver"; WO 01/08406 to Lambert et aL, pubUshed February 1, 2001, entitled "TV 
signal receiver"; and U.S. 2002/0131772 to Vrielink, published September 19, 2002, 
entitled "Methods of and devices for transmitting and reproducing audio and/or video 
information consisting of primary programs and conmiercials", and the aforementioned 

25 U.S. patent 6,177,960 to Van Luyt for a "TV signal receiver", each of which is 

incorporated herein by reference. Many of these techniques apply to programs having 
video and/or audio portions. 

A program recommender 160 provides information that indicates the degree of 
preference by a particular user, or a group of users, for different programs or shows - that 

30 is, the extent to which a user enjoys, or is expected to enjoy, watching a program. The 
preference can be provided for a currently watched program, or programs that are 
scheduled for future viewing, using any recommender technique. For example, one may 
use an explicit recommender, where a user indicates explicitly what aspects of programs 
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the user likes or dislikes. The user may indicate that he or she likes programs related to 
comedy, or specific sports events, or programs with specific actors, and so forth. Or, one 
may use an implicit recommender that learns the user's likes and dislikes fi-om the user's 
viewing/listening history. For example, the user can indicate via an interface that he likes 
5 or dislikes particular programs, and the recommender can extrapolate that information to 
determine whether the user would like or dislike other programs. The program 
recommender 160 may also predict the user's degree of preference for a program based on 
demographic information such as the user's gender, age, location, income and the like. 

For more information on program recommenders, see, e.g., U.S. patent application 

10 no. 09/466,406 to Srinivas Gutta, entitled "Method and Apparatus for Recommending 

Television Programming using Decision Trees," filed December 17, 1999 (Disclosure No. 
700772), and U.S. patent application no. 09/666,401 to Kaushal Kurapati, Dave Schaffer 
and Srinivas Gutta, entitled "Method and Apparatus for Generating Recommendation 
scores using Implicit and Explicit Viewing Preferences", filed September 20, 2000 (Filing 

15 No. US000239, Disclosure No. 701247), both of which are incorporated herein by 
reference. Many of these techniques apply to programs having video and/or audio 
portions. 

The commercial classifier 170 provides information to the CPU 140 regarding the 
degree of effectiveness of a commercial in relation to a particular program. Generally, this 

20 information is available fi"om advertisers, and reflects the degree of success of running the 
commercial in the particular program, e.g., based on resulting sales or sales inquiries, 
surveys, or other metrics. This information involves the effectiveness of the commercial in 
the particular program for all users. The present invention advantageously enables the 
effectiveness of a commercial to be determined for a particular user or a small group of 

25 users such as a family. 

Based on the information firom the program recommender 160 and the commercial 
classifier 170, the CPU 140 calculates an overall metric that is used to identify one or more 
commercials to display to the user in commercial breaks of a program currently being 
played on a display or other output device, or played at a future time. The commercials 

30 may be identified by a codeword identifier associated with each commercial or using any 
other scheme that is known in the art. When the commercials are stored in the commercial 
storage 120, they may be identified and located using any known memory management or 
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database storage techniques. See, for example, the aforementioned WO 98/36563, WO 
01/08406, U.S. 2002/0131772, and U.S. patent 6,177,960. 

Note that the configuration shown in Fig. 1 is a simplified example. Moreover, the 
various components that store and process information need not be distinct components but 
5 their fimctions can be combined and carried out by common processing and storage 
elements. 

Fig. 2 illustrates linking at least one user 205 to a number of programs 210, 212, 
214, 216, 218, ... and to a nimiber of commercials 260, 262, 264, 266, ... The program 
recommender 160 may provide the information that indicates the degree of preference of 

10 the user 205 for the programs 210, 212, 214, 216, 218, ... as numeric weights or scores 

denoted as w(user, show_t), where *V" denotes 'Veight", ^'user^* denotes a particular user, 
and show_t denotes a particular rth show, where t is an index representing each program. 
For example, the programs 210, 212, 214, 216, 218, ... may be denoted by t=l, 2, 3, 4, 5, 
... The weights may range between zero and one, for instance, indicating low and high 

15 preferences of the user 205 for a program. As an example, the user may have a preference 
of 0.9 for program 210, and a preference of 0.6 for program 212. Not all weights are 
shown. The preferences may be obtained fi-om an explicit and/or implicit recommender or 
other techniques, as discussed previously. 

The programs may be any one-time or series, e.g., recurrent, programs. For 

20 instance, program 210 may be the weekly news magazine "60 minutes", program 212 may 
be the weekly situation comedy program "Everybody loves Raymond", program 214 may 
be a bi-weekly "movie special," program 216 may be a daily re-run of "The Simpsons," 
and program 218 may be the daily program *The Tonight Show". 

Similarly, the commercial classifier 170 may provide the information that indicates 

25 the effectiveness of a commercial relative to a program as numeric weights or correlation 
factors denoted as w(show_t, comm_i), where "comm_i" denotes a particular ith 
commercial, where i is an index representing each commercial. For example, the 
commercials 260, 262, 264, 266, ... may be denoted by i=l, 2, 3, 4, ... , respectively. 
Moreover, t is an index representing each program. For example, the programs 210, 212, 

30 214, 216 and 218 ... may be denoted by t=l, 2, 3, 4, 5 respectively. The correlation 

factors may range between zero and one, for instance, indicating low and high correlations, 

respectively, of the commercial relative to a program. As an example, the commercial 260 

may have correlation factors of 0.3, 0.7, 0.1, 0.2 and 0.05 relative to programs 210, 212, 

5 
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214, 216, 218, respectively. Not all correlation factors are shown. Generally, the 
advertiser associated with a commercial can determine, e.g., from consumer surveys and 
other research, how strongly a commercial is linked to the target audience of each program, 
and obtain a corresponding correlation factor. The correlation factor of a commercial may 
5 indicate a retum on investment based on the dollar amount of sales generated from the 
commercial versus the amount of advertising dollars spent for the commercial time on a 
given program. Thus, the correlation factor for a commercial-program combination can be 
set by the advertiser, in one possible approach. This information can be communicated to 
the receiver 100 with a TV broadcast, for example, or using other techniques, such as by 

10 download via the Intemet. To provide a specific illustration, assimie the conmiercial 260 is 
for a particular coffee brand, commercial 262 is for a particular automobile, commercial 
264 is for a particular beverage, and commercial 266 is for a particular line of clothing. 

In accordance with the invention, the user-program weights or scores and the 
program-commercial correlation factors, are used to obtain a metric that correlates an 

15 effectiveness of the commercials to the individual user. This is achieved for each 

commercial by sxmiming the product of the weights and correlation factors over each 
program. For example, for the commercial 260, the metric is calculated as (0.9 x 0.3) + 
(0.6 X 0.7) + (0.3 X 0. 1) + (0.5 x 0.2) + (0.7 x 0.05) = 0.855. Generally, for each 
commercial, the metric that correlates an effectiveness of the commercials to the individual 

20 user/viewer can be calculated as: 

n— progr ams 

For all i: link(user, conrni i)= ^ w(user, show_t) • w(show_t, comm_i) 

Where n-programs is the number of programs. Stated alternatively, for each of the 
commercials, the degree of effectiveness (E) in relation to the viewer can be provided as: 

n— progr ams 

E= ^ score{t) • correlation factor{t) , where t is an index denoting each rth video 

25 program, where t=l, n-programs, score(t) denotes the score of the rth video program, 
and correlation factor(t) denotes the correlation factor relative to the rth video program. As 
one can see from the formula, a commercial gets a high degree of effectiveness if it has a 
high correlation factor to many programs that the user likes. For instance, sports 
commercials for yoimgsters can be correlated by the advertiser to both sports programs and 

30 programs for young people, and by the above formula the commercial will indeed get a 
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high computed effectiveness for people that like both kind of programs. The commercials 
with the highest metric values can be recommended for future display to the user. 

Fig. 3 illustrates an embodiment of a method for recommending commercials. The 
process starts at block 300 with the first commercial, and at block 305 with the first 
5 program. At block 310, the effectiveness metric (E) is initialized to zero. At block 320, a 
correlation factor (CF) is obtained indicating an effectiveness of the first commercial 
relative to the first program, ranging, e.g., fi-om zero for "least effective" to one for "most 
effective". At block 330, a score is obtained indicating the user's degree of preference for 
the current program, e.g., 0 for "hates", 0.1 for "strongly dislikes", 0.2 for "moderately 
10 dislikes", 0.5 for "neutral", 0.8 for "moderately likes", 0.9 for "strongly likes", and 
1.0 for "loves". At block 340, the effectiveness metric (E) is calculated fi-om E=E + 
(CF*S). 

At block 350, it is determined if there are programs remaining that have not been 
processed. If so, the correlation factor relative to the next program is obtained at block 

15 320, the score indicating the user's degree of preference is obtained at block 330, and the 
effectiveness metric is updated at block 340. Once the last program has been processed, as 
determined at block 350, the effectiveness metric (E) for the current commercial is stored. 
This is the final metric value for the current commercial. 

If there are additional commercials to process, as determined at block 370, the next 

20 commercial is processed starting at block 305 with the first program. At block 310, the 

effectiveness metric (E) is reset to zero. The process continues as discussed above until all 
commercials have been processed to obtain an effectiveness metric. At this time, one or 
more commercials are recommended for display to the viewer at block 380, e.g., based on 
the commercials with the highest effectiveness metrics. 

25 Note that the programs can be scored without determining which program is 

currently being played. A preference score can be predicted for each (future) show and a 
commercial can be recommended based on the effectiveness (E) metric, e.g., so that the 
commercials with the highest effectiveness metrics are recommended. Similarly, there is 
no need to monitor the commercials that are currently being played. Note that an identifier 

30 should be provided for each commercial for which a correlation factor is obtained so 
particular ones of the commercials that are recommended for display to a viewer can be 
easily identified. 
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When the personalized conunercials are displayed, they can be correlated to the 
programs in which they were previously run so that they run again in the same program. 
The program in which the personalized commercials run may be a subsequent presentation 
of a recurrent weekly program or in a similar type of program, e.g., programs in the 
5 category of situation comedies or sports events. Or, the personalized commercials need not 
be correlated to the programs in which they were previously nm, in which case they can be 
shown when the viewer views any subsequent program. Factors such as time of day or day 
of week can also be considered so that, e.g., a commercial that has run at a particular time 
of day and that is found to be particularly effective relative to a user can be run again at the 

10 same time of day in a subsequent day. For example, a coffee commercial may be more 
effective in the morning when more people drink coffee. 

Note also that the process of Fig. 3 can be completed for each user or for a group of 
users. For example, multiple users in a home may each be identified by an id nxmiber or 
other identifier that they provide when viewing programs via the user interface 130. In this 

15 way, the commercials can be personalized for the current user or a group of users (e.g., a 
family). For a group of user, step 330 may be modified to use an average of the 
preferences of the different users, such as a uniform average or a weighted average. 
Moreover, the process of Fig. 3 may be repeated from time to time to reflect changes in the 
scheduled programs and/or in the commercials. 

20 While there has been shown and described what are considered to be preferred 

embodiments of the invention, it will, of course, be understood that various modifications 
and changes in form or detail could readily be made without departing fi-om the spirit of the 
invention. It is therefore intended that the invention not be limited to the exact forms 
described and illustrated, but should be construed to cover all modifications that may fall 

25 within the scope of the appended claims. 



